Ranking using Metadata

This was the first time that RGU had participated in the HARD track, and indeed in TREC. We were interested in investigating the effect of exploiting the topic metadata to re-rank our initial baseline run, in a similar fashion to that of Rutgers in TREC 2003 [Belkin et al, 2003]. We used the Lemur toolkit (LTK) to obtain a baseline ranking, using title and description for each topic, and using OKAPI BM25 weighting (with default LTK settings). Then, we focussed on re-ranking this baseline for each topic, based on queries generated specifically to rank separately by genre, geography, and familiarity, using the LTK re-ranking capability (ranking of so-called “working set”). The baseline and metadata-derived rankings were then combined using an evidence combination approach.